Evolution of machine learning intelligence can take an extended amount of computing power, and requires a large amount of interaction with an accurate simulation of the environment or actual environmental interaction in order to explore potential solutions, find more novel cases in which to try to develop new solutions, and try tested solutions against. Parallel computation helps, but the knowledge may be different and represented in different ways in each individual computation, requiring that extra steps be taken to integrate the knowledge between the different parallel computations. Moreover, the operational environment may be dynamic and changing and may even be a Complex Adaptive System (CAS).
Machine learning systems tend to develop localized solutions that may not be shared. Genetics-based Learning Classifier Systems develop sets of rules that are tightly linked. The rules are tightly coupled through message lists, are in a careful balance of sensed and internal conditions, and can self modify as new knowledge is learned. Thus the rules are difficult to share between intelligent agents, and may be deleted when the program is stopped. The development of intelligent agents through machine learning requires a machine learning environment. This can be a large scale simulation, requiring at least one computer, or more. The results from one learning system currently cannot be shared with another.
Genetically programmed systems will create programs of operation that are influenced by the environment, whether a software-based environment or an external world sensed through some mechanism. These programs may be very detailed and specific to a particular model of robot or intelligent agent. The knowledge that it has learned that enabled it to write its own program and function at a high level of fitness may be saved only internally to the individual robot, or not at all. One such example is of a genetically programmed Learning Classifier Systems (LCS) for Complex Adaptive Systems (CAS) as described in U.S. Pat. No. 6,741,974 “Genetically Programmed Learning Classifier System for Complex Adaptive System processing with agent-based architecture” assigned to the assignee hereof. The disclosure of the '974 patent is hereby incorporated herein by reference. In CAS, agents are designed to perceive their environment through its detectors and take action on its perception through its effectors. Obviously, genetically programmed systems are different from systems where the robot or intelligent agent is programmed by a human, as the human can test and distribute the program as a software update. But the intelligent agent has to test and develop its software automatically, if it is equipped with machine learning.
A number of technologies exist for genetically programmed systems, machine learning, and distributed machine learning. For example, policy learning systems can learn a policy to go from state to state that can be shared, but this is limited to smaller, stable systems and is not suitable for large scale knowledge representation. GPU methods can speed operations when working on a data set with a fixed architecture, but are not amenable to automatically changing programs. Some searching algorithms are parallelizable to different machines, and the results grouped later. Other examples include: MLbase (T. Kraska, A. Talwalkar, J. Duchi, R. Griffith, M. Franklin, M. I. Jordan. MLbase: A Distributed Machine Learning System. In Conference on Innovative Data Systems Research, 2013; and A. Talwalkar, T. Kraska, R. Griffith, J. Duchi, J. Gonzalez, D. Britz, X. Pan, V. Smith, E. Sparks, A. Wibisono, M. J., Franklin, M. I., Jordan. MLbase: A Distributed Machine Learning Wrapper. In Big Learning Workshop at NIPS, 2012), U.S. Patent Application Nos. 20060224535, 20060248031, and U.S. Pat. Nos. 6,741,974, and 8,200,610. These solutions generally distribute raw data or knowledge or search through it. However, none of these solutions provide for an actionable program that is capable of joining with other local programs to create a higher fitness program, nor do they provide for continuously-adaptable genetic-programming-based learning, especially at the Complex Adaptive Systems (CAS) level.
If intelligent agents could share their memories and incorporate knowledge from other intelligent agents then actionable knowledge learned in one location could be transferred to a second location to increase the capabilities of the agent at the second location. Generally, knowledge is learned in detail, and the ability to abstract is a higher-level intelligence function that is not generally ascribed to machine learning. But it is exactly that type of generalized and abstract knowledge along with individualized behaviors that would be helpful in a distributed learning approach. Accordingly, there is a need in the art for distributed machine learning that will share the information and incorporate the information from other intelligent agents, and provide for a distributed diverse array of intelligent agents to develop more robust operational code, and to develop the code in a parallel manner to make the genetic evolutionary process faster. There is a further need in the art for a process that allows parallel execution of the machine learning exploration phase to enhance the speed and breadth of machine learning.
The embodiments herein are designed to address these needs.